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ARI  Research  Reports  and  Technical  Reports  are  intended  for  sponsors  of 
R&O  tasks  and  for  other  research  and  military  agencies.  Any  findings  ready 
for  implementation  at  the  time  of  publication  are  presented  in  the  last  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recom¬ 
mendations  for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


The  Leadership  and  Management  Technical  Area  of  the  Army  V  ...earch  Insti¬ 
tute  for  the  Behavioral  and  Social  Sciences  (AR1)  conducts  research  address¬ 
ing  the  issues  of  Army  leader  assessment,  training,  and  performance.  Since 
nearly  75%  of  new  officers  commissioned  each  year  are  trained  through  the 
Army  Reserve  Officers'  Training  Corps  (ROTC),  research  on  the  ROTC  training 
program  has  great  potential  for  enhancing  officer  performance. 

In  1983,  the  Deputy  Chief  of  Staff  for  ROTC  initiated  the  Achievement 
Testing  Program  to  assess  the  levels  of  basic  skills  of  ROTC  cadets  with  the 
objective  of  developing  standardized  diagnostic  and  selection  procedures. 

ARI  was  asked  to  perform  the  data  analysis. 

This  report  describes  the  results  of  the  analyses  in  terms  of  the  distri¬ 
bution  of  achievement  test  scores  for  Military  Science  I  (MS  I)  and  MS  IV 
classes  as  a  whole  and  for  subgroups  of  cadets.  The  report  also  indicates 
additional  types  of  information  needed  for  ROTC  to  formulate  diagnostic  and 
selection  standards  as  well  as  the  organizational,  social,  and  political 
issues  that  must  be  addressed  in  this  process. 


EDGAR  M.  JOHNSON 
Technical  Director 


ROTC  ACHIEVEMENT  TESTING  PROGRAM:  SCHOOL  YEARS  1983-1985 


EXECUTIVE  SUMMARY 


Requirement : 

The  1978  report  of  the  Review  of  Education  and  Training  of  Officers  (RETO) 
recommended  a  standardized  screening  procedure  to  be  established  for  precommis- 
sionlng  programs.  The  Deputy  Chief  of  Staff  for  ROTC  (DCSROTC),  Training  and 
Doctrine  Command  (TRADOC),  initiated  a  project  to  assess  basic  skills  of  ROTC 
cadets.  The  ODCSROTC  requested  the  Army  Research  Institute  (ARI)  to  analyze 
achievement  test  data  for  School  Years  1983-85  and  to  report  findings  that 
might  be  used  by  ROTC  to  develop  diagnostic  and  screening  standards. 


Procedure : 

ROTC  administered  the  Missouri  College  English  Test,  the  Nelson-Denny 
Reading  Test,  and  the  Stanford  Achievement  Test:  High  School  Mathematics 
Test  to  Military  Science  I  (MS  I)  and  MS  IV  cadets.  Nearly  the  entire  popu¬ 
lation  of  MS  IV  cadets  participated,  while  the  participation  rate  for  MS  I 
was  a  little  better  than  50%.  However,  for  all  classes  tested,  the  propor¬ 
tional  representation  by  region  and  gender  closely  approximated  the  percent¬ 
age  breakdowns  for  these  subgroups  of  cadets  reported  by  the  ROTC  Headquarters 
for  the  respective  school  years  and  MS  classes. 

This  report  is  based  on  data  from  School  Years  (SYe)  1983-84  and  1984-85. 
The  distributions  of  test  scores  were  examined  within  each  MS  class  by  gender, 
racial/ethnic  group,  and  academic  major. 


Findings : 

Females  scored  higher  than  males  in  English;  males  scored  higher  than 
females  in  mathematics;  and  there  were  no  substantial  gender  differences  in 
reading  scores.  However,  on  all  tests,  the  ethnic  minority  groups  scored 
lower  than  the  white  group;  the  differences  were  substantial  for  the  black 
and  Hispanic  groups.  The  average  scores  of  MS  I  cadets  were  consistently 
lower  than  MS  IV  scores  possibly  due  to  attrition  of  lower  achieving  students, 
selection  applied  at  contracting  between  MS  II  and  MS  III,  and/or  greater 
experience  and  education  of  MS  IV  cadets.  Based  on  scores  that  might  be  used 
as  possible  cut  scores  for  MS  IV,  the  analyses  identified  about  20%  of  MS  I 
cadets  who  might  profit  from  remedial  training  in  English  and  reading. 
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The  academic  majors  were  grouped  as  engineering/architecture,  physical 
sciences,  social  sciences,  and  "other."  The  first  three  groups  scored  higher 
than  the  other  group  in  all  tests.  The  first  three  groups  did  not  differ 
greatly  in  reading  and  writing  skills,  though  the  social  science  group  scored 
lower  than  the  other  two  groups  in  mathematics. 


Utilization  of  Findings: 

The  data  from  this  project  provide  comparisons  of  average  test  scores 
among  subgroups  of  cadets.  However,  within  the  scope  of  this  project,  it  was 
not  possible  to  examine  the  relationship  between  precommissioning  measures  of 
academic  achievement  and  officer  job  performance.  In  order  to  ensure  adequate 
levels  of  basic  skills  of  ROTC  graduates  and  to  meet  the  increasing  officer 
production  requirement,  further  research  and  considerations  of  organizational, 
social,  and  political  issues  are  needed. 
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ROTC  Achievement  Testing  Program: 
School  Years  1983  -  1985 


Background 


An  important  aspect  of  precommissioning  officer  training  is  to  ensure  that 
ail  newly-commissioned  Army  officers  have  attained  an  adequate  level  of  basic 
skills.  The  Army  Reserve  Officers'  Training  Corps  (ROTC)  produces  nearly  75% 
of  new  officers  commissioned  each  year,  and  this  production  mission  is 
expected  to  increase  in  the  coming  years.  The  basic  skills  assessment  and 
development  effort  in  the  ROTC  program  could  make  an  important  contribution  to 
ROTC  training  effectiveness  as  well  as  officer  utilization. 

The  1978  report  of  the  Review  of  Education  and  Training  of  Officers  (RETO) 
recommended  that  a  standardized  method  be  established  to  screen  and  evaluate 
officer  potential  of  candidates  for  precommissioning  programs.  The 
recommendation  was  followed  by  the  Precommissioning  Assessment  System  (PAS) 
(U.S.  Department  of  the  Army,  1978)  which  identified  five  components 
considered  to  be  important  to  military  leadership:  academic,  physical, 
medical,  psychological,  and  leadership  potential.  PAS  also  established 
initial  guidelines  for  assessing  each  of  these  components  for  the  purpose  of 
admitting,  or  contracting,  cadets  into  ROTC  Advanced  Course. 

The  only  standard  currently  used  to  assess  cadets'  academic  competence 
for  commissioning  is  a  college  degree  and  grade  point  average  of  2.0  or  above. 
However,  grading  standards  vary  greatly  among  universities  and  colleges. 

Thus,  a  need  for  further  efforts  in  refining  evaluation  standards  was 
reiterated  in  the  1984  report  of  Army  Science  Board  Summer  Study  (U.S. 
Department  of  the  Army,  1984). 

Responding  to  this  need,  the  Deputy  Chief  of  Staff  for  ROTC  (DCSROTC) , 
the  Training  and  Doctrine  Command  (TRADOC),  initiated  a  project  in  1983  to 
establish  diagnostic  and  screening  standards  for  standardized  achievement 
tests.  The  plan  was  to  administer  achievement  tests  during  three  consecutive 
school  years  to  ROTC  cadets  in  Military  Science  I  (MS  I)  and  Military  Science 
IV  (MS  IV)  classes.  Based  on  the  configuration  of  the  scores  obtained  from 
this  initial  phase  of  the  project,  ROTC  was  to  formulate  a  standardized  method 
of  assessing  and  developing  basic  skills.  ODCSROTG  requested  the  Army 
Research  Institute  (ARI)  to  analyze  the  achievement  fast  data  for  School  Years 
(SY)  1983--86  to  provide  a  description  of  cadet  performance  which  would  be  used 
by  ROTC  In  setting  diagnostic  and  screening  standards. 
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Objectives  of  the  Achievement  Testing  Program 


The  Achievement  Testing  Program  was  designed  to  serve  two  purposes: 

1.  To  establish  guidelines  for  diagnosing  basic  skills  deficiencies 
among  cadecs  in  the  ROTC  Basic  Course  (MS  I  and  II  or  the 
purpose  of  directing  those  with  marginal  or  poor  p.  ‘ormance  to 
receive  remedial  training. 

2.  To  set  criteria  for  proficiency  in  reading,  writing,  and 
mathematics  for  admissions  to  the  Advanced  Course  of  ROTC  (MS 
111  and  IV)  and  for  commissioning. 

The  immediate  objective  for  the  achievement  testing  in  school  years  1983 
through  1986  was  to  establish  an  empirical  basis  for  formulating  preliminary 
standards  for  diagnosis,  screening,  and  commissioning  which  might  be  adjusted 
when  test  results  from  subsequent  years  were  examined. 

Purpose  of  Report 

The  purpose  of  this  report  is  to  present  results  from  the  achievement  test 
data  analysis.  It  will  describe  test  performance  of  ROTC  cadets  as  a  whole, 
comparisons  among  gender  and  ethnic  subgroups,  and  how  many  cadets  passed 
various  score  points. 

Setting  selection  standards  Is  a  highly  complex  process.  Besides  the  test 
scores,  many  other  issues  must  be  taken  into  consideration  such  as  the 
relationships  between  basic  skills  and  officer  job  requirements,  the  number  of 
new  officers  needed,  resources  needed  to  continue  testing,  and  the  social/ 
political  Implications  of  using  cut  scores  for  selection.  Much  of  the 
information  needed  to  address  these  issues  is  currently  lacking,  requiring 
further  empirical  research. 

The  results  presented  in  this  report  will  provide  one  of  numerous  pieces 
of  information  required,  namely  the  test  performance  of  ROTC  cadets.  However, 
they  alone  will  not.  be  sufficient  to  specify  sound  guidelines  for  selection  or 
diagnostic  procedures. 


Procedure 


Achievement  Tests  Used  in  the  Project 

ROTC  selected  three  commercially  available  achievement  tests  to  be  used 
for  this  project.  Each  test  provides  national  “norms,"  l.e.,  performance  of  a 
large  sample  of  students  across  geographical  areas,  as  a  basis  for  comparison. 

Missouri  College  English  Test  (Callis  &  Johnson,  1965).  This  test  was 
designed  to  assess  knowledge  of  “the  mechanics  and  effectiveness  of  written 
expression"  (Callis  &  Johnson,  1965,  p.  3).  It  is  a  90-item  multiple-choice 
test,  requiring  no  actual  writing.  The  test  was  standardized  based  on  data 


from  college  freshmen  attending  universities  and  two-  and  four-year  colleges, 
both  private  and  public,  from  all  regions  of  the  United  States,  during  the 
early  1960s. 

Two  issues  need  to  be  addressed  in  using  this  test:  (1)  The  national 
norm  was  established  for  college  freshmen  only  since  the  test  was  intended  as 
a  diagnostic  tool  to  identify  freBhman  students  in  need  of  remedial  training 
in  writing.  When  the  test  is  administered  to  more  advanced  college  students 
in  higher  college  grades,  as  in  the  ROTC  Achievement  Testing  Program,  the 
interpretation  of  raw  scores  according  to  the  national  norms  may  be  less 
accurate.  Older  students  may  be  likely  to  score  higher,  due  to  greater 
experiences  and  learning  opportunities  and/or  student  attrition,  part  of 
which  may  be  based  on  lack  of  basic  skills. 

(2)  The  original  norming  sample  was  adequate  in  number  and  regional  and 
type-of-school  representation.  However,  this  was  the  only  standardization 
performed  for  this  test  on  a  national  sample.  Since  then,  educational 
achievement  levels  have  shifted  (McGeever,  1983).  It  may,  or  may  not,  be 
accurate  to  judge  writing  abilities  of  the  80s  students  (even  college 
freshmen)  against  the  performance  of  freshmen  in  the  early  60s.  As  described 
in  the  subsequent  section,  these  problems  were  partially  dealt  with  by 
computing  ROTC  norms  based  on  the  data  collected  for  this  project. 

The  Nelson-Penny  Beading  Test  (Brown,  Bennett,  &  Hanna,  1981).  This 
test  provides  assessment  of  vocabulary  knowledge,  reading  comprehension, 
and  reading  rate.  The  two  sub  tests  (Vocabulary,  Comprehension)  can  be  used 
selectively  to  diagnose  students'  weak  areas.  Scores  from  both  subtests 
combined  (Total)  indicate  their  overall  reading  achievement. 

The  national  norms  for  this  test  were  developed  in  the  late  70s,  based  on 
scores  from  students  in  9th  through  16th  grades  (high  school  and  college). 
Separate  norms  are  published  for  each  of  the  8  grades  included,  so  that,  for 
example,  a  college  senior's  score  can  be  compared  to  a  national  sample  of 
college  seniors.  Interpreting  the  ROTC  students'  performance  against  the 
national  norm  is  more  accurate  with  this  test  than  with  the  Missouri  English 
test,  because  the  standardization  was  done  fairly  recently  and  separate  norms 
for  each  college  grade  are  available. 

Stanford  Achievement  Teat;  High  School  Mathematics  Test  (Gardner,  Merwin, 
Callis,  &  Madden,  1965).  This  test  consists  of  three  parts:  a  Numeric 
Competence  subtest,  which  assesses  students'  general  numeric  and  arithmetic 
capabilities;  a  Mathematics  Part  A  subtest,  which  contains  problems  covered  in 
high  school  elementary  algebra  and  geometry  textbooks,  and  a  Mathematics  Part 
B  subtest,  which  tests  knowledge  of  advanced  algebra,  trigonometry,  and  some 
new  mathematics  concepts  from  high  school  textbooks.  For  the  purpose  of 
diagnosis  for  remediation,  the  test  writers  suggest  using  the  Numeric 
Competence  sub  test  score. 

The  two  problems  mentioned  with  reference  to  the  Missouri  English  test 
also  apply  to  this  test.  The  test  was  standardized  based  ou  students  iu  high 
school  grades  (9th  through  12th);  the  reference  group  which  is  closest  to  ROTC 
cadets  is  the  college  preparatory,  high  school  seniors.  In  additiou,  the 


norming  data  were  collected  la  the  early  60s.  Thus,  Interpretiag  the 
performance  of  ROTC  cadets,  especially  college  juniors  and  seniors,  according 
to  these  norms  may  not  be  very  accurate. 

Reliability  and  validity  of  the  achievement  tests.  The  reliability  of 
these  tests  reported  In  the  manuals  are  all  in  the  .90s  range  and  indicate 
adequate  stability  of  scores  on  these  tests.  Table  1  presents  the  reliability 
coefficients  and  methods  of  deriving  them  for  each  test.  The  content 
validity,  i.e.,  the  degree  to  which  test  items  cover  the  subject  matter,  waB 
investigated  by  test  developers  for  each  test  and  reported  to  be  satisfactory. 
However,  the  question  of  how  accurately  these  tests  may  predict  college 
graduates'  job  performance,  specifically  in  the  military  setting,  has  not  been 
addressed  empirically. 


Table  1 

Reliabilities  of  Achievement  Tests* 


Reliability  Sample  Method 


Test 

Coefficient 

Grade 

Number 

llBed 

Missouri  College 

.94 

13th 

12,580 

Split-half 

English  Test 

corrected  by 
Spearman-Brown 

Nelson- Denny 

.93 

13th  & 

239 

Alternate  forms 

Reading  Test 
(Total  Bcores) 

14  th 

correlations 

S  tanf ord-Ma thema tics 

12  th 

4,897 

Split- half 

Numeric  Competence 

.92 

corrected  by 

Mathematics  Total 

.94 

Spearman-Brown 

Extracted  from  Callis  and  Johnson  (1965)  for  English,  Brown  et  al.  (1981 ' 


for  reading,  and  Gardner  et  al.  (1965)  for  mathematics  tests. 


Test  Administration  Procedure 

The  ROTC-vide  administration  of  the  three  achievement  tests  for  the 
purpose  of  developing  diagnostic  and  screening  standards  was  carried  out 
over  three  consecutive  school  years  beginning  with  SY  83-84.  During  SY  83-84, 
the  English  and  the  reading  tests  were  administered  in  the  spring  semester  to 
MS  IV  classes  of  mostly  juniors  and  seniors.  In  addition,  the  reading  test 
was  given  to  MS  I  cadets,  mostly  freshmen.  In  SY  84-85,  the  English  and  the 
reading  tests  were  administered  to  MS  I  and  IV  classes  in  the  spring.  The 
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mathematics  test  was  administered  in  the  Advanced  Camp  duriug  the  summers  of 
1984  and  1985,  In  the  last  year  of  the  research  project  (SY  85-86),  data  for 
English  and  reading  were  collected  from  MS  1  and  MS  IV  cadets  in  the  fall. 


This  report  is  based  on  data  from  the  first  two  years.  Table  2  presents 
the  data  collection  schedule,  the  military  science  classes  included,  and  the 
number  of  cadets  who  participated  in  the  testing. 

Tests  ware  administered  by  the  cadre  in  each  ROTC  detachment  and  then 
forwarded  to  the  test  publishers  for  scoring.  The  coded  data  on  magnetic 
tapes  were  sent  to  AR1  from  ROTC  Headquarters  for  analysis. 


Table  2 

ROTC  Achievement  Testing  Program 
Testing  Schedule  and  Number  of  Cadets  Tested 


Tests  and 

Time  of  Administration 

MS  1 

Class 

MS  III  MS  IV 

SY  83-84  Spring  Semester 

Nelson-Denny  Reading 

20,218 

7,524 

Missouri  English 

7,365 

Summer  Advanced  Camp 

Stanford  Mathematics 

8,221 

SY  84-85  Spring  Semester 

Nelson-Denny  Reading 

18,930 

7,306 

Missouri  English 

16,001 

7,558 

Summer  Advanced  Camp 

Stanford  Mathematics 

8,265 

Demographic  Description  of  Cadets  Participating 

in  the 

Achievement  Testing 

Program 

Tables  3  and  4  present  the  total  number  of 

cade  ts 

taking  each  of  the 

achievement  tests  during  the  two  school  yearB  and  the  percentages  of  subgroups 
by  ROTC  region,  gender,  racial/ethnic  groups,  and  academic  major  categories. 


Table  3  describes  the  MS  I  students  who  took  the  English  test  in  SY  34-85 
and  the  reading  test  in  SYs  83-84  and  84-85.  The  to  til  number  of  cadets 
included  in  each  of  these  data  sets  represents  approximately  half  of  the 
end-of-school-year  MS  1  enrollment  for  the  respective  years.  The  breakdown 
of  cadets  across  ROTC  regions  reflect  the  regional  distribution  of  ROTC 
detachments;  most  of  the  cadets  attend  schools  in  Regions  1,  11,  and  Ill. 

(See  Appendix  1  for  the  states  Included  in  each  ROTC  region.)  For  the  test 
data,  the  regional  representation  remained  fairly  stable  over  the  two  years 
with  one  exception  of  some  decrease  in  cadets  from  Region  III  in  SY  84-85. 

Cadets  identified  themselves  into  one  of  the  following  ethnic  groups: 

(1)  Asian/Pacific  Islander,  (2)  black  -  Negro,  not  of  Hispanic  origin, 

(3)  Hispanic,  (4)  white  -  Caucasian,  not  of  Hispanic  origin,  (5)  American 
Indian/native  Alaskan,  and  (6)  Other.  In  the  analyses  and  tables  reported 
below,  the  American  Indian/native  Alaskan  group,  representing  about  .5%  of 
every  data  set,  was  combined  with  the  Other  group.  In  termB  of  racial/ethnic 
backgrounds,  approximately  73%  of  MS  1  cadets  were  white,  21%  black,  3% 
Hispanic,  and  1.5%  Asian/Pacific  Islanders.  For  both  school  year3,  about  28% 
of  the  MS  1  cadets  were  females. 

The  mathematics  test  was  administered  in  Advanced  Camp  to  cadets  who  had 
completed  MS  111  classes  and  ware  to  begin  MS  IV  in  the  fall.  For  the  sake  of 
simplicity,  the  Advanced  Camp  cadets  who  took  the  mathematics  test  and  the  MS 
IV  cadets  who  took  the  English  and  the  reading  tests  will  be  designated  "MS 
IV"  cadets  In  the  remainder  of  this  report. 

The  MS  IV  data  setB  included  much  higher  proportions  of  the  end-of-school- 
year  enrollment  figures,  ranging  from  88  to  100%  with  a  mean  of  96%.  The 
largest  proportion  of  MS  IV  cadets  tested  came  from  Region  i  followed  by 
Regions  II,  III,  and  IV,  conforming  to  the  regional  distributions  reported  In 
the  ROTC  Enrollment  Reports.  Compared  to  the  MS  1  data,  the  relative 
representation  of  the  MS  IV  cadets  from  Region  IV  was  considerably  greater, 
while  the  relative  representation  of  Region  111  cadets  was  smaller. 

Percentages  of  males  and  females  in  the  MS  IV  data  remained  stable  over 
the  two  school  years  at  approximately  16%  females  and  84%  males.  However, 
compared  to  the  MS  1  data,  the  relative  proportion  of  females  was  reduced  to 
nearly  half,  from  about  28%  to  about  16%.  This  suggests  that  a  large  number 
of  female  cadets  might  not  continue  in  the  ROTC  program  at  the  contracting 
time. 

The  racial/ethnic  backgrounds  of  the  MS  IV  cadets  were:  73-76%  white, 

1 6--1 8%  black,  4-5%  Hispanic,  and  2%  Asian-Pacific  Islanders.  The  proportion 
of  the  black  cadets  was  lower,  and  that  of  the  Hispanics  was  slightly  higher, 
than  in  MS  1  classes. 


Table  3 

Demographic  Composition  of  MS  I  Cadets  Participating 
in  the  Achievement  Testing  Program  (Percentage  of  Total  Tested) 


Reading  data 

Reading  data 

V 

English  data  |a 

Group 

SY  83-84 

SY  84-85 

SY  84-85  N 

N-20,218 

N-18,930 

N-16,001  p| 

£ 

ROTC  Region  p 

Region  1 
Region  11 
Region  111 
Region  IV 


Gender 

Female 

Male 


Raclal/Ethnlc  Group 

Asian 

Black 

Hispanic 

White 

Other 


Academic  Majors 
Engineer^  Architect 
Physical  Science 
Social  Science 
Medical 
Other 


29 

32 

29 

32 

25 

26 

7 

7 

8 

27 

28 

28 

73 

72 

72 

1 

2 

2 

22 

21 

20 

3 

4 

3 

72 

73 

73 

2 

1 

2 

13 

12 

12 

15 

14 

14 

12 

13 

12 

7 

7 

8 

54 

55 

54 

B 


Table  4 

Demographic  Composition  of  MS  III  and  MS  IV  Cadets  Participating 
in  the  Achievement  Testing  Program  (Percentage  of  Total  Tested) 


Group 

Reading 
SY  83-84 
N-7,524 

Reading 
SY  84-85 
N-7,306 

English 
SY  83-84 
N-7,365 

English 
SY  84-85 
N-7,558 

Math 

SY  83-84 
N-8,221 

Math 

SY  84-85 
N-8,265 

ROTC  Region 

Region  I 

43% 

44% 

42% 

43% 

39% 

43% 

Region  11 

21 

25 

22 

24 

25 

24 

Region  III 

19 

17 

20 

18 

20 

19 

Region  IV 

16 

15 

16 

14 

16 

15 

Gender 

Female 

16 

16 

16 

17 

15 

15 

Male 

84 

84 

84 

83 

85 

85 

Racial/Ethnic  Group 


Asian 

2 

2 

2 

2 

2 

2 

Black 

18 

16 

18 

16 

18 

18 

Hispanic 

4 

6 

5 

4 

5 

5 

White 

75 

76 

73 

76 

73 

73 

Other 

1 

1 

2 

2 

2 

2 

Academic  Majors 
Engineer/ Archi tec  t 

9 

12 

9 

11 

11 

12 

Physical  Science 

15 

16 

14 

15 

15 

16 

Social  Science 

32 

30 

31 

28 

28 

27 

Medical 

4 

4 

4 

5 

4 

4 

Other 

40 

38 

41 

41 

42 

41 

The  proportions  of  cadets  by  academic  majors  are  also  indicated  in  Tables 
3  and  4.  Academic  majors  were  grouped,  guided  by  the  grouping  procedure  used 
by  the  Military  Personnel  Center  (MILPERCEN)  Officer  Distribution  Division  as 
follows:  Engineering/architecture,  physical  science  (computer  science, 
physical  science,  natural  science,  math/ statistics),  social  science  (social 
science,  psychology),  medical  (pre-med,  nursing),  other  (business/accounting, 
management,  education,  letters,  fine  arts,  and  undeclared). 

The  engineer/architecture  group  made  up  about  10%  of  MS  I  and  IV  classes. 
Likewise,  the  percentage  of  cadets  in  physical  sciences  was  fairly  consistent, 
at  about  15%  of  all  classes  tested.  On  the  other  hand,  the  percentage  of 
social  science  majors  differed;  about  12%  for  MS  1  classes  and  about  30%  for 
MS  IV  classes.  The  ''Other''  group  constituted  the  majority,  55%  for  MS  I 
classes  and  about  40%  for  MS  IV  classes.  The  pre-med  and  nursing  students 
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were  combined,  even  though  their  academic  programs  may  differ  substantially, 
since  together  they  make  up  less  than  10%  of  the  ROTC  population  (basic  and 
advanced  courses),  and  many  of  them  are  commissioned  through  the  special 
appointment  avenue. 

To  summarize  the  demographic  composition  of  the  cadets  participating  in 
the  achievement  testing,  the  proportions  representing  the  ROTC  regions, 
genders,  and  racial/ethnic  groups  remained  fairly  constant  over  the  two  school 
years.  Nearly  the  entire  population  of  MS  IV  cadets  participated.  For  MS  1 
testing,  the  participation  rate  was  a  little  better  than  50%.  However,  for 
all  of  these  classes  tested,  the  proportional  representation  by  region  and 
gender  closely  approximated  the  percentage  breakdowns  reported  by  the  ROTC 
Headquarters  for  the  respective  school  years  and  MS  classes.  The  differences 
between  the  sample  subgroup  percentages  and  the  percentages  reported  in  ROTC 
Closing  Enrollment  Reports  ranged  from  0  to  5  percent,  averaging  at  1.6 
percent.  Comparing  the  demographic  compositions  between  the  MS  I  and  IV 
classes,  some  marked  differences  were  noted.  The  proportions  of  females  and 
black  cadets  were  lower  in  MS  IV  than  in  MS  I. 


Results 


ks 


In'. 


Data  used  for  analysis  were:  the  total  score  for  the  Missouri  College 
English  Test,  the  total  of  the  Vocabulary  and  Comprehension  subtestB  for  the 
Nelson-Denny  Reading  Test,  and  the  Numeric  Competence  subteBt  score  for  the 
Stanford  Achievement  Test  of  Mathematics. 


The  results  of  the  analysis  are  presented  in  terms  of  the  actual  score  a 
cadet  achieved  on  each  test  (raw  score)  and  the  ROTC  and  national  percentile 
scores.  The  raw  scores  are  not  comparable  across  the  three  achievement  tests; 
the  maximum  possible  scores  for  English,  reading,  and  mathematics  are  90,  172, 
and  45,  respectively.  Since  a  given  raw  score  (e.g.,  138  on  the  reading  test) 
in  itself  does  not  denote  a  certain  level  of  competence,  one  might  refer  to 
the  percentile  scores  based  on  national  samples.  A  percentile  score  indicates 
the  percentage  of  people  who  scored  at  or  below  a  given  raw  score.  However, 
using  the  published  percentile  scores  of  these  tests  to  infer  ability  levels 
may  not  be  appropriate  since  the  English  and  the  mathematics  tests  were  normed 
on  test  scores  of  college  freshmen  and  high  school  seniors,  respectively,  both 
In  the  early  1960s. 

The  ROTC  percentile  scores  provide  current,  age-appropriate  norms  based  on 
the  same  ROTC  reference  group  across  tests.  Using  the  ROTC  percentiles,  it  Is 
possible  to  compare  the  relative  positions  of  various  subgroups,  and  also  to 
compare  individual  performance  across  the  three  tests.  For  example,  an 
English  raw  score  of  37,  a  reading  score  of  92,  and  a  mathematics  score  of  25 
all  correspond  to  the  20th  ROTC  percentile  score  for  MS  IV  cadets  for  the 
respective  tests.  This  means  that  20  percent  of  MS  IV  cadets  scored  below 
these  raw  scores  on  these  tests.  On  the  other  hand,  despite  the  limitations 
of  the  published  national  percentile  scores,  they  do  provide  some  basis  on 
which  to  compare  the  ROTC  percentile  scores  against  the  performance  of 
students  in  general.  For  this  purpose,  both  percentile  scores  are  presented 
in  some  of  the  tables. 


The  score  distributions  were  highly  similar  over  the  two  school  years,  as 
exemplified  in  Table  5.  For  the  remainder  of  the  report,  tabular  and  written 
descriptions  of  the  results  will  be  based  mainly  on  the  SY  83"84  data.  The  MS 
IV  test  results  will  be  presented  first,  followed  by  the  MS  1  test  results. 

MS  IV  Achievement  Test  Results 


This  section  describes  the  results  of  the  mathematics  testing  in  Advanced 
Camp  (in  the  summer  between  the  junior  and  the  senior  years)  and  the  English 
and  the  reading  testing  of  MS  IV  cadets  in  the  spring.  Analysis  of  these  data 
was  related  to  the  issue  of  standards  for  contracting  or  commissioning,  it 
was  guided  by  the  question  of  what  scores  might  be  used  as  selection  criteria 
which  would  serve  to  retain  the  maximum  number  of  cadets  who  possess 
sufficient  basic  skills  to  perform  their  duties  as  Army  officers.  The 
analyses  were  also  directed  at  the  differential  effects  that  various  cut 
scores  might  have  on  gender  and  ethnic  subgroups. 

It  must  be  emphasized  here  that  the  results  reported  below  are  not 
Intended  to  suggest  or  recommend  any  selection  standard.  The  particular 
ranges  of  scores  examined  in  the  following  sections  were  chosen  only  for 
illustrative  purposes,  i.e.,  to  show  the  patterns  of  score  distributions  for 
the  whole  group  and  for  subgroups. 

Missouri  Collage  Engliah  Test.  For  this  test,  the  maximum  raw  score  is 
90,  and  the  national  norm  is  based  on  college  freshmen  of  the  early  60s. 

As  shown  in  Table  5,  the  average  raw  score  for  the  total  group  was  50  for  both 
years.  Taking  the  SY  83-84  data,  the  average  for  females  was  romewhat  higher 
than  for  males,  54  and  49,  respectively.  However,  more  substantial 
differences  were  found  among  the  racial/ethnlc  groups.  The  black  and  Hispanic 
groups  scored  substantially  lower  than  the  white  group. 

Table  6  shows  the  number  of  cadets  in  these  ethnic  groups  who  scored  at 
or  above  given  scores  on  the  English  test.  The  figures  in  columns  labeled  n 
indicate  the  actual  numbers  of  cadets  scoring  at  or  above  a  given  score;  the 
percentage  lu  parentheses  indicates  the  proportion  of  the  subgroup  (e.g., 
white,  black)  that  scored  at  or  above  this  score.  The  "total"  column 
summarizes  the  results  across  the  ethnic  groups, 

A  raw  score  of  37  on  the  English  test  corresponds  to  the  20th  MS  IV 
percentile  point  (meaning  that  approximately  80%  of  the  total  MS  IV  group 
scored  at  or  above  this  score).  For  SY  83-84,  882  white,  47%  black,  58% 
Hispanic,  and  75%  Asian  cadets  achieved  a  score  of  37  or  higher.  In  other 
words,  20%  of  the  total  MS  IV  cadets  failed  to  achieve  this  score,  but  some 
ethnic  groups  were  represented  disproportionately  in  this  20%. 
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Table  5 

Average  Raw  and  Percentile  Scores  for 
Missouri  College  English  Test,  MS  IV,  SY  83-84  and  SY  84-85 


SY 

83-84 

SY 

84-35 

Raw 

Score 

SD* 

Percentile  Scores 
ROTC  National 

Raw 

Score 

SD* 

Percentile  Scores 
ROTC  National 

Total 

50 

15 

49.6 

40.2 

50 

15 

49.5 

40.4 

Gender 

Male 

49 

14 

48.3 

38.9 

50 

15 

48.4 

39.3 

Female 

54 

15 

56.5 

46.7 

53 

15 

55.2 

45.7 

Ethnic  Group 

Aslan 

48 

14 

44.8 

36.0 

49 

15 

46.2 

37.4 

Black 

38 

13 

26.5 

20.6 

38 

13 

23.0 

20.4 

Hispanic 

41 

15 

33.3 

26.3 

43 

14 

35.0 

27.8 

White 

54 

13 

56.8 

46.2 

53 

13 

55.6 

45.5 

Other 

40 

16 

42.9 

34.4 

48 

15 

45.6 

36.9 

*  Standard  deviation  of  raw  scores. 


Table  6 

Cumulative  Number  and  Percentage  of  MS  IV  Cadets  Scoring  Above 
Various  Scores  on  Missouri  College  English  Test  by  Ethnic  Group 


tercenttle 
Raw  |  Scores 

Score |R0IC  /  National 

White 

Black 

Hispanic 

Asian 

Other 

Total 

n 

(X) 

n 

(X) 

n 

(X) 

n  (%) 

n  ( 

;x> 

a  (X) 

SY  83 

-  84 

37 

20 

15 

4737 

(88) 

629 

(47) 

213 

(58) 

116  (75) 

101 

(70) 

5796  (79) 

34 

15 

11 

4929 

(92) 

747 

(56) 

240 

(66) 

121  (79) 

109 

(76) 

6146  (84) 

30 

10 

7 

5116 

(95) 

914 

(68) 

270 

(74) 

137  (89) 

116 

(81) 

6553  (89) 

25 

5 

4 

5269 

(98) 

1092 

(82) 

309 

(84) 

147  (95) 

128 

(89) 

6945  (94) 

21 

2 

2 

5323 

(99) 

1190 

(89) 

330 

(90) 

149  (97) 

133 

(92) 

7125  (97) 

Total 

5359  (100) 

1336  (100) 

366  (100) 

154  (100) 

144  (100) 

7359  (100) 

NOTE: 

Hie 

"total"  is  less 

than  the 

total  number 

tasted  due  to  missing  data. 
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Nelson -Depay  Read log  Teat.  The  possible  raw  score  range  for  this  test  is 
from  0  to  172.  However,  the  national  norm  provides  a  fairly  accurate  basis  to 
interpret  the  raw  scores  relative  to  the  performance  of  a  national  sample 
since  the  norms  were  developed  for  each  college  grade  in  early  1980s. 

The  means  for  the  total  ROTC  group  remained  stable  over  the  two  years  at 
the  raw  score  of  119.  Table  7  shows  that  the  average  scores  for  males  and 
females  were  very  similar  for  this  test,  although  large  differences  were  found 
among  the  ethnic  group  means. 


Table  7 


Average  Raw  and  Percentile 
Nelson-Denny  Reading  Test,  MS 

Scores 
IV,  SY 

for 

83-84 

Raw 

Score 

SD 

Percentile  Scores 
ROTC  National 

Total 

119 

30 

49. 5 

43.0 

Gender 

Male 

119 

30 

49.9 

43.3 

Female 

117 

32 

48.6 

42.4 

Ethnic  Group 

ABlan 

no 

30 

40.7 

34.3 

Black 

87 

26 

21.1 

16.0 

Hispanic 

96 

34 

30.8 

25.6 

White 

128 

24 

57.5 

50.8 

Other 

119 

32 

49.9 

43.9 

Standard  deviation  of  raw  scores. 


Table  8  displays  the  number  of  cadets,  by  ethnic  group,  passing  various 
score  points.  For  SY  83-84,  at  the  ROTC  20th  percentile  (raw  score  of  92), 

91 Z  of  white  cadets  passed,  but  39%  and  55%  of  the  black  and  Hispanic  cadets, 
respectively,  passed.  The  performance  of  ethnic  minority  groups  was  generally 
poorer,  when  compared  to  the  total  group,  on  the  Nelson-Denny  Reading  Test 
than  on  the  Missouri  English  test. 
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Table  8 

Cumulative  Number  and  Percentage  of  MS  IV  Cadets  Scoring 
Above  Various  Scores  on  Nelson-Denny  Reading  Test  by  Ethnic  Group 


►'t 

hi 


SY  83  -84 


92 

20 

14 

5078 

(91) 

509 

(39) 

177 

(55) 

103 

(72) 

70 

(74) 

5977 

(79) 

86 

15 

10 

5275 

(94) 

623 

(47) 

193 

(60) 

111 

(79) 

77 

(82) 

6315 

(84) 

77 

10 

5 

5437 

(97) 

784 

(60) 

220 

(68) 

119 

(84) 

80 

(85) 

6677 

(89) 

64 

5 

2 

5547 

(99) 

1031 

(75) 

252 

(78) 

129 

(91) 

83 

(94) 

7086 

(94) 

54 

2 

1 

5591 

(99) 

1161 

(88) 

277 

(85) 

133 

(94) 

91 

(97) 

7294 

(97) 

Total 

5607 

(100) 

1312 

(100) 

324 

(100) 

141 

(100) 

94 

(100) 

7524 

(100) 

Stanford  Mathematics  Test.  As  Table  9  shows,  the  overall  raw  score  mean 
on  this  test  was  3i  (with  the  possible  score  range  of  0  to  42).  The  national 
norms  on  this  test  are  not  very  informative  since  they  were  developed  in  early 
60s  baaed  on  performance  of  college-preparatory  high  school  seniors.  Males 
performed  considerably  better  than  the  females  on  this  test.  For  SY  83-84, 
the  respective  ROTC  percentile  means  were  51.3  and  39.4.  Table  9  also  shows 
the  same  pattern  of  ethnic  group  differences  found  with  the  other  testB: 
blacks  and  Hispanics  scored  substantially  lower  than  whites. 

Table  10  indicates  that  about  80%  of  the  total  ROTC  group  scored  above  a 
raw  score  of  25.  However,  this  group  included  87%  of  the  whites,  842  oi  the 
Asians,  44%  of  the  blacks  and  61%  of  the  Hispanics. 
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Table  9 

Average  Raw  and  Percentile  Scores  for 
Staaford  Math  Test,  MS  III,  SY  83-84 


% 


& 


Raw  Score 

SD* 

R0TC 

National  S 

Total 

32.1 

8.1 

49.5 

43.2 

Gender  l 

Male 

32.6 

8o0 

Jl.3 

44.9 

Female 

29.0 

8.5 

39.4 

33.2  £ 

Ethnic  Group  ; 

Asian 

33.1 

7.2 

52.4 

45.8 

Black 

24.7 

7.6 

24.6 

19.0 

Hispanic 

28.1 

8.0 

35.3 

29.0  t 

White 

34.2 

7.0 

56.8 

50.4 

Other 

30.9 

8.3 

45.1 

38.6  fc 

*  Standard  deviation  of  raw  scores, 


Table  10 

Cumulative  Number  and  Percentage  of  MS  III  Cadets  Scoring  Above 
Various  Scores  for  Stanford  Mathematics  Teat  by  Ethnic  Group 


White  Black  Hispanic  Aslan  Other  Total 

Raw  |  Percentile  Scores 

Scorej  ROIC  National  n  (X)  n  (%)  n  (X)  n  (X)  n  (X)  n  (X) 

______ 


25 

23 

14 

5231 

(87) 

663 

(44) 

271 

(61) 

128 

(84) 

112 

(74) 

6405 

(78) 

23 

15 

10 

5456 

(91) 

7% 

(53) 

307 

(69) 

136 

(90) 

122 

(81) 

6817 

(83) 

20 

10 

6 

5712 

(96) 

1018 

(68) 

356 

(81) 

140 

(93) 

132 

(87) 

7358 

(90) 

17 

5 

4 

5847 

(98) 

1228 

(82) 

397 

(90) 

145 

(96) 

139 

(92) 

7756 

(94) 

14 

2 

2 

5924 

(99) 

1350 

(90) 

422 

(95) 

150 

(99) 

144 

(95) 

7990 

(97) 

Total 

5979 

(100) 

1492 

(100) 

442 

(100) 

151 

(100) 

151 

(100) 

8215 

(100) 

Note:  'hie  ,HIbtal"  la  less  than  the  total  number  tested  due  to  missing  data. 


The  gender  and  ethnic  group  scores  across  three  tests.  There  were  gender 
differences  in  two  of  the  tests;  females  scored  higher  in  English  and  lower  in 
mathematics  than  males.  The  average  of  males'  English  scores  was  not  far 
below  the  total  average,  mainly  because  males  constituted  close  to  852  of  the 
total  group.  However,  the  females'  average  mathematics  score  was  considerably 
below  that  of  the  males. 

The  ethnic  minority  groups,  especially  the  blacks  and  the  Hispanics, 
scored  substantially  lower  than  the  white  group  in  all  skill  domains  tested. 
For  all  ethnic  minority  groups,  the  percentage  of  cadets  passing  the  20th 
percentile  point  was  the  lowest  for  the  reading  test.  For  black  cadets,  392, 
442,  and  472  passed  the  20th  ROTC  percentile  points  of  the  reading, 
mathematics,  and  English  tests,  respectively. 

Analysis  of  composite  scores  based  on  three  tests.  The  objective  of  the 
Achievement  Testing  Program  was  to  determine  basic  skillB  qualification 
standards.  However,  the  realities  of  officer  production  mission  sud  the 
extent  of  differential  impact  of  selection  standards  on  subgroups  must  be 
considered  in  this  process.  An  individual's  performance  on  any  given  test  can 
fluctuate,  so  that  it  is  not  judicious  to  base  a  critical  decision  concerning 
his  or  her  future  career  on  one  test  score.  In  addition,  many  individuals  do 
not  have  the  same  degree  of  competence  in  all  skill  areas,  but  rather  perform 
better  in  one  area  than  another. 

Table  11  shows  the  similarities  of  individual  performance  on  two  tests, 
e.g.,  between  English  and  reading  scores.  The  higher  the  correlations,  the 
more  similar  the  performance  on  two  tests.  For  the  total  group,  English  aud 
reading  test  scores  are  highly  correlated  (r  -  .70),  but  the  associations 
between  English  and  mathematics  and  be tween~reading  and  mathematics  ace  only 
moderate  (r  ■  ,55  and  .51,  respectively).  Many  of  the  correlations  between 
the  verbal~tests  (English  and  reading)  and  mathematics  are  in  the  .406  range 
for  ethnic  subgroups.  These  results  suggest  that  some  cadets  might  score 
above  the  cut  score  (e.g.,  40th  ROTC  percentile)  on  one  test  but  below  the  cut 
score  on  another,  largely  due  to  varying  expertise  in  different  skill  areas. 


Table  11 

Correlations  Between  Achievement  Test  Scores 


English 

:md 

Reading 

English 

and 

Math 

Reading 

and 

Math 

Total 

.70 

.55 

.51 

White 

.63 

.46 

.38 

Black 

.61 

.45 

.40 

Hispanic 

.68 

.53 

.52 

Aslan 

.78 

.48 

CO 

• 

Other 

.71 

.77 

.63 

Note:  ROTC  percentile  scores  were  used  for  these  analyses.  Nath  data  from 
SY  83  -  84,  MS  111;  English  and  reading  d«ta  from  SY  84  -  85,  MS  IV 
classes. 


If  several  test  scores  are  used  to  evaluate  cadets'  overall  competence  in 
basic  skills,  their  strengths  in  one  area  may  compensate  for  marginal  scores 
in  another.  This  effect  is  shown  on  Table  12,  which  displays  (in  the  first 
three  columns)  the  percentage  of  minority-group  cadets  passing  the  10th  and 
the  20th  ROTC  percentile  scores  for  the  three  tests  separately.  The  fourth 
column  contains  the  percentages  of  cadets  passing  these  points  based  on  each 
cadet's  average  of  the  English  and  the  reading  ROTC  porcentlle  scores.  The 
last  column  shows  the  percentages  based  on  averages  of  each  cadet's  ROTC 
percentile  scores  on  all  3  teats. 


16 


Table  12 

Percentage  cf  Minority-Group  Cadets  Exceeding  the  10th  and  the  20th 
ROTC  Percentile  Points  for  Each  Test  and  Combinations  of  Tests 


English  English  Reading 

English  Reading  Math  and  and  and  All 


10th  Percentile 

Black 

66% 

63% 

68% 

69% 

75% 

72% 

75% 

Hispanic 

79 

63 

81 

83 

88 

83 

88 

Asian 

89 

81 

93 

93 

98 

95 

96 

20th  Percentile 

Black 

46 

40 

44 

48 

50 

49 

50 

Hispanic 

59 

46 

61 

66 

77 

67 

77 

Asian 

75 

67 

84 

75 

89 

86 

87 

Comparing  the  results  for  separate  teats  and  those  for  averaged  data 
indicate  that  the  number  of  minority  cadets  who  would  pass  a  given  cut  point 
would  increase  if  multiple  test  scores  are  considered  together.  For  example, 
if  the  20th  percentile  score  on  the  English  test  alone  were  used  as  a 
selection  standard,  only  46%  of  the  black  cadets  would  pass.  If  composite 
scores  based  on  all  tests,  or  English  and  mathematics,  were  used,  50%  would 
pass.  For  the  black  group,  which  constitutes  about  20%  of  the  total  MS  IV 
population,  the  difference  of  4%  or  5%  currently  amounts  to  50-60  more  cadets 
to  be  commissioned. 

Based  on  general  knowledge  about  ability  testing  and  the  empirical 
illustration  made  above,  it  eaems  that  using  a  composite  score  for  a  selection 
procedure  would  have  the  following  advantages: 

1)  Multiple  test  scores  are  more  reliable  than  one  to  represent  an 
Individual's  overall  abilities. 

2)  Since  many  individuals  have  different  areas  of  strengths  and  since 
a  basic  level  of  quantitative  skills  would  be  needed  for  many 
officer  jobs,  both  skill  areas  should  be  assessed. 

3)  If  the  English  find  mathematics  scores  are  included  in  the 
composite,  the  number  of  cadets  passing  the  cut  score  is  likely  to 
increase. 


4)  By  increasing  Che  number  of  cadets  passing,  this  approach  would 
reduce  Che  differential  impact  of  selection  process  across  the 
ethnic  minority  groups. 

However,  currently  there  is  no  sound  information  on  how  well  any  of  the 
separate  tests  or  composite  scores  would  predict  performance  in  Army  officer 
jobs.  Analyses  reported  above  suggest  that  additional  research  needed  to 
clarify  these  questions  is  warranted. 

MS  I  Achievement  Test  Results 

The  purpose  for  administering  the  English  and  the  reading  tests  to  MS  I 
cadets  was  bo  examine  the  score  distributions  for  new  cadets  which  would,  in 
turn,  provide  a  basis  for  establishing  remediation  guidelines.  As  with  the  MS 
IV  data,  it  was  not  possible  to  determine  what  level  of  reading  and  writing 
abilities  a  given  test  score  represented  since  there  was  no  other  measure  of 
these  skills  with  which  the  achievement  test  scores  could  be  compared. 

Given  this  constraint,  the  following  steps  were  taken:  1)  compare  raw 
scores  of  MS  1  and  IV  cadets  and  2)  examine  the  number  of  MS  1  cadets 
performing  near  the  score  which  may  be  used  as  a  cut  score  for  selection 
(contracting  and/or  commissioning).  Raw  scores  were  used  for  comparison  since 
percentile  scores  of  MS  1  and  MS  TV  groups  are  based  on  different  reference 
groups  and  are  not  comparable.  The  following  comparisons  are  based  on  SY 
84-85  data  which  included  English  and  reading  tests  given  to  both  MS  I  and  IV 
classes. 
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Table  13  shows  tha  average  raw  scores  of  MS  I  sad  MS  IV  classes  on  English 
and  reading  tests  for  each  class  as  a  whole,  by  gender,  and  by  ethnic  group. 
The  MS  XV  averages  are  consistently  higher  than  the  MS  £  averages. 


Table  13 

MS  1  and  MS  IV  Raw  Score  MeanB 
(SY  84  -  85  Data) 


English  Readini 


MS  I 

MS  IV 

MS  I 

MS 

IV 

Total 

46  (15) 

50  (15) 

101  (32) 

119 

(30) 

Gender 

Male 

45  (15) 

50  (15) 

102  (32) 

120 

(30) 

Female 

48  (16) 

53  (15) 

99  (32) 

117 

(32) 

Ethnic  Group 

Aslan 

47  (15) 

49  (15) 

101  (35) 

til 

(32) 

Black. 

35  (13) 

37  (13) 

73  (26) 

80 

(27) 

Hispanic 

37  (16) 

43  (14) 

.  76  (35) 

93 

(34) 

White 

50  (14) 

53  (13) 

110  (28) 

128 

(24) 

Other 

43  (16) 

48  (15) 

95  (34) 

121 

(33) 

Note;  Score  range  for  the  English  test  is  0  -  90. 

Score  range  for  the  reading  test  is  0  -  172. 
Standard  deviations  are  indicated  in  parentheses. 


Table  14  displays  the  percentages  of  MS  1  cadets  scoring  at  or  above  a 
hypothetical  cut  score  for  MS  iV  cadets.  The  ruw  scores  of  37  and  93,  for  the 
English  and  the  reading  tests,  respectively,  were  the  scores  at  which  20X  of 
MS  IV  cadets  in  SY  84-*85  failed.  This  table  also  shows  a  smaller  percentage 
of  MS  1  cadets  passing  these  hypothetical  cut  score  points  on  both  tests, 
across  ethnic  groups. 


Table  14 

Percentages  of  MS  1  and  MS  IV  Cadets  Passing  a  Given  Score 
on  Missouri  English  and  Nelson-Denny  Reading  Tests 
by  Ethnic  Group  (SY  84  -  85) 


Raw  Score 

White 

Black 

Hispanic 

Aslan 

Other 

Total 

English 

MS  I 

37 

802 

38% 

46% 

71% 

61% 

69% 

MS  IV 

37 

87% 

46% 

59% 

75% 

72% 

78% 

Reading 

MS  I 

93 

71% 

20% 

31% 

55% 

50% 

58% 

MS  IV 

93 

90% 

40% 

46% 

67% 

75% 

79% 

Note:  English  raw  score  of  37  and  reading  raw  score  of  93  axe  the  scores  that 
about  SOX  of  MS  XV  cadets  passed  in  SY  84  -  85. 


It  is  difficult  to  determine  why  the  MS  IV  grouj,>  scored  higher  than  the  MS 
I  group,  especially  since  the  data  for  the  two  groups  are  not  longitudinal 
(i.e.,  not  collected  from  the  same  group  of  people  at  two  points  in  time). 

Part  of  the  difference  may  reflect  attrition  of  low  achievers  from  colleges 
and  ROTC  programs  betvean  MS  £  and  MS  IV  years.  The  attrition  may  be 
voluntary  or  due  to  selection  applied  at  contracting  time.  The  difference  may 
also  reflect  more  years  of  education  And  greater  expertise  on  the  pert  of  the 
older  cadets. 


If  these  changes  ere  Lhe  primary  reasons  for  the  score  differences  between 
MS  1  and  MS  IV  cadets  and  if  ROTC  ware  commiseioning  the  required  number  of 
new  officers  each  year,  then  a  special  effort  for  remediation  training  may  not 
be  necessary.  However,  currently  there  is  a  need  to  Increase  the  number  of 
commisslonees.  If  remediation  programs  could  improve  the  basic  skills  of 
marginally  achieving  cadets,  it  would  enlarge  the  pool  of  cadets  available  for 
commissioning. 


An  approach  for  establishing  remediation  guidelines.  As  mentioned 
earlier,  there  are  no  data  available  to  determine  the  relationship  between  the 
achievement  test  scores  end  performance  in  officer  jobs.  In  addition,  since 
the  Achievement  Testing  Program  date  are  not  longitudinal,  the  degree  to  which 
MS  I  students  might  increase  their  basic  skills  without  special  remediation 
training  is  not  known.  Given  this  situation,  the  only  feasible  way  to  develop 
preliminary  guidelines  for  early  remediation  might  be  to  use  scores  which 
might  be  chosen  as  selection  criteria  as  reference. 
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As  aa  Illustrative  example,  suppose  that  a  cut  score  is  set  at  the  raw 
score  at  which  80S  of  MS  IV  cadets  pass.  (Depend log  on  the  size  of  cadet  pool 
and  officer  production  requirements,  the  cut  score  might  be  higher  or  lower.) 
Even  though  the  overall  performance  of  the  MS  1  cadets  was  lower  than  that  of 
MS  IV  classes,  it  is  assumed  that  by  the  time  of  selection  (for  contracting  or 
commissioning),  a  cadet  would  be  expected  to  pass  the  cut  score  point.  It  is 

also  assumed  that  those  who  scored  somewhat  above  the  cut  score  at  time  1  (say 

In  MS  1  class)  may  not  pass  at  a  later  time,  since  any  individual's 
performance  on  a  given  test  can  fluctuate  over  multiple  besting.  Further, 
those  who  barely  make  the  cut  score  would  benefit  from  developing  the  basic 
skill.  On  the  other  hand,  those  who  score  somewhat  lower  than  the  cut  score 
may  also  Increase  their  skills  through  special  training  and  pass  the  test  the 
second  time . 

The  question  is  what  proportion  of  MS  1  cadets  fall  in  the  score  range  of 
"somewhat  above"  and  "somewhat  below"  the  cut  score.  For  English  and  reading 
data  from  SY  84-85,  the  MS  I  cadets  were  grouped  into  these  score  categories, 

using  as  the  "cut  score  point"  the  score  where  80S  of  MS  IV  cadets  passed. 

Table  15  shows  the  percentage  breakdowns  by  teBt. 


Table  15 


Percentage  of  MS  1 
Close  to  a  Possible 

Cadets  in  Score  Ranges 

Cut  Score  (SY  84  -  85) 

Raw  Score  Range 

Percent 

English 

44  and  above 

57 

37  -  43 

15 

30  -  36 

12 

29  and  below 

17 

Reading 

101  and  above 

50 

93  -  100 

9 

80  -  92 

14 

61  -  79 

15 

60  and  below 

12 

Note:  English  raw  score  of  37  and  reading  raw  score  of  93  are  the 
scores  that  80%  of  MS  IV  cadets  passed  in  SY  84-85. 
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For  English,  80%  of  MS  IV  cadets  passed  the  raw  score  of  37  (the  NS  IV 
average  was  50);  About  15%  of  MS  I  cadets  scored  37-43.  About  12%  of  MS  I 
scored  30  to  36.  A  score  of  30  Is  at  about  the  10th  percentile  point  for  MS 
IV  cadets  and  would  be  an  extremely  low  score  as  a  selection  standard.  Of 
course,  students  falling  in  the  very  low  score  category  may  possibly  improve 
skills  to  pass  the  cut  score,  with  good  instructions  and  high  motivation.  But 
it  appears  particularly  beneficial  for  cadets  in  the  middle  groups  (somewhat 
above  and  below  the  cut  point)  to  be  strongly  encouraged  to  seek  remedial 
training. 

For  the  Nelson-Denny  reading  test,  the  hypothetical  cut  score,  based  on 
80%  pass  for  MS  IV  Cadets  (SY  84-85),  is  93.  The  MS  IV  average  is  119.  The 
score  range  that  might  be  considered  for  remediation  training  is  from  80  to 
100.  About  23%  of  MS  I  cadets  fall  in  this  middle  range.  On  this  test,  about 
12%  of  MS  I  cadets  scored  61  or  below,  which  is  the  range  where  about  5%  of  MS 
IV  cadets  scored.  It  may  be  difficult  for  most  students  scoring  in  this  range 
to  improve  their  reading  skills  enough  to  pass  92. 

Comparison  of  Average  Scores  by  Academic  Majors 

Table  16  presents  average  raw  and  ROl'C  percentile  scores  for  MS  1  and 
MS  IV  cadets  by  test  and  by  academic  major  groups  (i.e.,  engineering/ 
architecture,  physical  science,  social  science,  medical,  and  other).  Note 
score  averages  allow  comparisons  between  MS  1  and  IV  cadets,  while 
percentiles  Indicate  the  position  of  a  given  subgroup  relative  to  the 
class.  The  means  of  MS  IV  class  were  based  on  data  from  SY  83-84  and 
classes  combined. 


that  raw 
i  the  R0TC 

I  total  MS 

(  SY  84-85 
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Table  16 

Average  Raw  and  ROTC  Percentile  Scores  by  College  Major 


Engineer/ 

Architect 


Physical 

Science 


Social 

Science 


Medical 


Other 


Missouri  English 


MS  IV 
Raw  Score 
ROTC  Percentile 

Nelson-Denny  Reading 


MS  I V 

Raw  Score 
ROTC  Percentile 
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52 

52 

51 

54 

48 

53 
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50 
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121 
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52 

52 

53 

55 

44 
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(16) 

(27) 

(14) 

(42) 

37 

34 

30 

33 

31 

68 

57 

43 

55 

45 

(12) 

(14) 

(12) 

(08) 

(54) 

50 

48 

48 

49 

44 

57 

53 

53 

54 

45 

(13) 

(15) 

(12) 

(07) 

(54) 

110 

103 

106 

103 

96 

58 

52 

54 

52 

45 

Note:  Numbers  in  parentheses  indicate  percentage  of  each  major  group  in 
respective  MS  class. 

Medical  category  includes  pre-med  and  nursing  majors. 


For  both  MS  1  and  IV  classes,  average  scoreB  on  the  English  test  were 
fairly  close  among  the  engineering,  physical,  and  social  science  groups;  the 
group  averages  for  all  of  these  groups  were  50th  or  above  RCTC  percentiles. 

The  "Other"  group,  which  makes  up  the  majority,  scored  lower.  For  MS  IV 
cadets,  the  reading  averages  of  all  groups  except  "Other"  were  very  close,  and 
substantially  higher  than  the  "Other"  average.  This  pattern  is  generally 
repeated  for  the  MS  1  data. 
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The  mathematics  test  was  given  only  to  MS  III  cadets,  and  no  basic  and 
advanced  course  comparison  was  possible.  However,  this  test  revealed  more 
striking  differences  among  the  academic  major  groups.  Engineering/ 
architecture  students  scored,  as  a  group,  substantially  higher  than  the 
physical  science  majors,  who  in  turn  scored  higher  than  the  social  science  and 
"Other”  groups. 

Recall  that  the  main  difference  in  the  acAdemic  major  compositions  between 
MS  I  and  IV  classes  was  that  the  percentage  of  social  science  mnjors  was 
greater  in  MS  IV  than  in  MS  I.  In  MS  IV,  this  group  performed  as  well  as  the 
engineering  and  physical  science  groups  on  English  and  reading  tests. 


Summary  and  Conclusions 

The  ROTC  Achievement  Testing  Program  was  designed  to  serve  two  purposes: 

1)  to  establish  guidelines  for  diagnosing  weaknesses  in  basic  Bkills  and 

2)  to  establish  selection  standards. 

There  were  several  constraints  in  the  research  design.  Only  achievement 
test  data  were  collected,  with  no  other  measure  of  academic  achievement  to 
verify  the  meaning  of  scores  on  these  teats.  Coupled  with  outdated  and/or 
age-inappropriate  national  norms,  interpretation  of  the  "goodness"  of  any 
given  score  was  difficult.  Different  year  groups  of  cadets  were  involved  in 
testing  each  year  so  that  changes  from  MS  1  to  MS  IV  years  could  not  be 
inferred. 

Some  of  the  major  findings  from  the  Achievement  Testing  Program  are 
summarized  below.  However,  with  the  lack  of  many  other  essential  data,  the 
findings  reported  In  this  report  must  be  considered  preliminary  and  partial 
with  reference  to  the  Issues  of  diagnostic  and  selection  standards. 

ROTC  norma  enabled  comparisons  of  achievement  levels  among  subgroups.  For 
both  MS  I  and  IV  classes,  gender  differences  were  relatively  minor,  but  there 
were  considerable  disparities  among  ethuic  groups  across  all  tests.  Using 
composite  scores,  derived  from  averaging  two  or  more  test  scores,  may  provide 
a  way  to  assess  overall  achievement  level  across  various  academic  domains,  to 
increase  some  cadets'  chances  of  meeting  the  selection  criteria,  and  to 
contribute  towards  ROTC  officer  production  mission. 

About  202  of  MS  1  cadets  scored  somewhat  above  or  somewhat  below  a  score 
that  was  arbitrarily  selected  as  a  possible  cut  score  for  selection.  If  these 
cadets  are  informed  of  selection  standards  early  in  their  college  career  and 
seek  additional  training  in  their  weaker  areas,  this  may  have  a  significant 
impact  on  the  overall  quality  and  the  number  of  ROTC-trained  commissionees. 

Analyses  of  academic  majors  snowed  that  the  majority  of  cadets  major  In 
subjects  other  than  engineering/architecture,  physical,  and  social  sciences 
and  that  they,  as  a  group,  scored  lower  than  these  groups.  The  engineering/ 
architecture  and  the  two  science  groups  did  not  differ  greatly  in  reading  and 
writing  skills,  though  the  social  science  group  scored  lower  than  the  other 
two  groups  in  mathematics. 
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Ensuring  the  basic  competency  of  new  officers  is  an  essential  mission  of 
all  preconunissioning  programs.  However,  the  Army  ROIC  also  has  a  mission  to 
increase  the  number  of  commissionees  during  the  next  several  years  when  a 
decline  is  expected  in  college  enrollment.  For  example,  ROTC  projects 
commissioning  7,500  new  officers  in  FY87  with  the  requirement  of  9,200  (U.S. 
Department  of  the  Army,  1986,  in  preparation).  Even  with  a  basic  skills 
selection  standard  which  disqualifies  the  lowest  10%  of  the  prospective 
commissionees,  the  rate  of  production  shortfall  would  increase  from  18%  to 


In  order  to  approach  systematically  the  multiple  objectives  of  ROTC, 
additional  empirical  research  is  needed.  First,  there  must  be  a  clear 
knowledge  of  which,  and  how  much,  basic  skills  are  needed  for  Army  officer 
jobs.  Then,  the  effectiveness  of  basic  skills  development  programs  needs  to 
be  assessed.  Issues  to  be  addressed  Include:  Which  basic  skills  are  amenable 
to  developmental  procedures?  Which  method  la  most  effective  and 
coat-effective?  What  levels  of  deficiencies  are  most  likely  to  be  improved? 
What  are  the  rate  and  permanence  of  change? 

One  such  effort  was  undertaken  within  the  military  context  during  the  60s 
and  early  70s.  The  focus  of  this  project  was  on  functional  literacy  among  the 
enlisted  personnel  and  effectiveness  of  an  experimental  reading  training 
program  which  was  integrated  into  the  military-job-skills  training.  In  terms 
of  job-related  reading  skills,  a  substantial  improvement  was  obtained  (Sticht, 
1975a,  1975b). 

Unfortunately,  these  findings  are  not  directly  applicable  to  college 
population,  curriculum,  and  officer  Job  requirements.  In  addition,  although 
basic  skills  remediation  programs  are  commonly  offered  at  moat  colleges  and 
universities,  systematic  information  concerning  their  effectivenass  is  scarce. 
Thus,  examining  the  effectiveness  of  developmental  procedures  seems 
warranted. 

Finally,  decisions  regarding  selection  standards  also  must  be  based  on 
officer  job  analysis  and  knowledge  of  predictive  relationships  between  various 
basic  skills  measurements  and  officer  job  performance. 
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APPENDIX  1 


ROTC  Region  Jurisdiction 


First  ROTC  Region,  Fort  Bragg,  NC  28307 


Connecticut  Maine 

Delaware  Maryland 

District  of  Columbia 
Florida  Hew  Hampshire 

Georgia  New  Jersey 


New  York 
North  Carolina 
Massachusetts 
Rhode  Island 
South  Carolina 


Vermon  t 
Virginia 
Pennsylvania 
Puerto  Rico 
Virgin  Islands 


Illinois 

Indiana 


Alabama 

Arkansas 


Alaska 

Arizona 

California 

Colorado 

Hawaii 


Second  ROTC  Region,  Fort  Knox,  KY  40121 

Kentucky  Missouri 

Michigan  Ohio 

Third  ROTC  Region,  Fort  Riley,  KS  66442 

Kansas  Mississippi 

Louisiana  New  Mexico 


Fourth  ROTC  Region 

Idaho 

Iowa 

Minnesota 

Montana 

Nebraska 


Fort  Lewis,  WA  98433 
Nevada 

North  Dakota 
Oregon 
South  Dakota 
Utah 


Tennessee 
West  Virginia 
Wisconsin 


Oklahoma 

Texas 


Washington 

Wyoming 

Guam 

American  Samoa 


